927 research outputs found
An exact solution of spherical mean-field plus orbit-dependent non-separable pairing model with two non-degenerate j-orbits
An exact solution of nuclear spherical mean-field plus orbit-dependent
non-separable pairing model with two non-degenerate j-orbits is presented. The
extended one-variable Heine-Stieltjes polynomials associated to the Bethe
ansatz equations of the solution are determined, of which the sets of the zeros
give the solution of the model, and can be determined relatively easily. A
comparison of the solution to that of the standard pairing interaction with
constant interaction strength among pairs in any orbit is made. It is shown
that the overlaps of eigenstates of the model with those of the standard
pairing model are always large, especially for the ground and the first excited
state. However, the quantum phase crossover in the non-separable pairing model
cannot be accounted for by the standard pairing interaction.Comment: 5 pages, 1 figure, LaTe
Towards Ultra-High Performance and Energy Efficiency of Deep Learning Systems: An Algorithm-Hardware Co-Optimization Framework
Hardware accelerations of deep learning systems have been extensively
investigated in industry and academia. The aim of this paper is to achieve
ultra-high energy efficiency and performance for hardware implementations of
deep neural networks (DNNs). An algorithm-hardware co-optimization framework is
developed, which is applicable to different DNN types, sizes, and application
scenarios. The algorithm part adopts the general block-circulant matrices to
achieve a fine-grained tradeoff between accuracy and compression ratio. It
applies to both fully-connected and convolutional layers and contains a
mathematically rigorous proof of the effectiveness of the method. The proposed
algorithm reduces computational complexity per layer from O() to O() and storage complexity from O() to O(), both for training and
inference. The hardware part consists of highly efficient Field Programmable
Gate Array (FPGA)-based implementations using effective reconfiguration, batch
processing, deep pipelining, resource re-using, and hierarchical control.
Experimental results demonstrate that the proposed framework achieves at least
152X speedup and 71X energy efficiency gain compared with IBM TrueNorth
processor under the same test accuracy. It achieves at least 31X energy
efficiency gain compared with the reference FPGA-based work.Comment: 6 figures, AAAI Conference on Artificial Intelligence, 201
CAG: A Real-time Low-cost Enhanced-robustness High-transferability Content-aware Adversarial Attack Generator
Deep neural networks (DNNs) are vulnerable to adversarial attack despite
their tremendous success in many AI fields. Adversarial attack is a method that
causes the intended misclassfication by adding imperceptible perturbations to
legitimate inputs. Researchers have developed numerous types of adversarial
attack methods. However, from the perspective of practical deployment, these
methods suffer from several drawbacks such as long attack generating time, high
memory cost, insufficient robustness and low transferability. We propose a
Content-aware Adversarial Attack Generator (CAG) to achieve real-time,
low-cost, enhanced-robustness and high-transferability adversarial attack.
First, as a type of generative model-based attack, CAG shows significant
speedup (at least 500 times) in generating adversarial examples compared to the
state-of-the-art attacks such as PGD and C\&W. CAG only needs a single
generative model to perform targeted attack to any targeted class. Because CAG
encodes the label information into a trainable embedding layer, it differs from
prior generative model-based adversarial attacks that use different copies
of generative models for different targeted classes. As a result, CAG
significantly reduces the required memory cost for generating adversarial
examples. CAG can generate adversarial perturbations that focus on the critical
areas of input by integrating the class activation maps information in the
training process, and hence improve the robustness of CAG attack against the
state-of-art adversarial defenses. In addition, CAG exhibits high
transferability across different DNN classifier models in black-box attack
scenario by introducing random dropout in the process of generating
perturbations. Extensive experiments on different datasets and DNN models have
verified the real-time, low-cost, enhanced-robustness, and high-transferability
benefits of CAG
Embedding Compression with Isotropic Iterative Quantization
Continuous representation of words is a standard component in deep
learning-based NLP models. However, representing a large vocabulary requires
significant memory, which can cause problems, particularly on
resource-constrained platforms. Therefore, in this paper we propose an
isotropic iterative quantization (IIQ) approach for compressing embedding
vectors into binary ones, leveraging the iterative quantization technique well
established for image retrieval, while satisfying the desired isotropic
property of PMI based models. Experiments with pre-trained embeddings (i.e.,
GloVe and HDC) demonstrate a more than thirty-fold compression ratio with
comparable and sometimes even improved performance over the original
real-valued embedding vectors
Beneath Surface Similarity: Large Language Models Make Reasonable Scientific Analogies after Structure Abduction
The vital role of analogical reasoning in human cognition allows us to grasp
novel concepts by linking them with familiar ones through shared relational
structures. Despite the attention previous research has given to word
analogies, this work suggests that Large Language Models (LLMs) often overlook
the structures that underpin these analogies, raising questions about the
efficacy of word analogies as a measure of analogical reasoning skills akin to
human cognition. In response to this, our paper introduces a task of analogical
structure abduction, grounded in cognitive psychology, designed to abduce
structures that form an analogy between two systems. In support of this task,
we establish a benchmark called SCAR, containing 400 scientific analogies from
13 distinct fields, tailored for evaluating analogical reasoning with structure
abduction. The empirical evidence underlines the continued challenges faced by
LLMs, including ChatGPT and GPT-4, in mastering this task, signifying the need
for future exploration to enhance their abilities.Comment: Accepted to EMNLP 2023 (Findings
Cluster Contrast for Unsupervised Person Re-Identification
State-of-the-art unsupervised re-ID methods train the neural networks using a
memory-based non-parametric softmax loss. Instance feature vectors stored in
memory are assigned pseudo-labels by clustering and updated at instance level.
However, the varying cluster sizes leads to inconsistency in the updating
progress of each cluster. To solve this problem, we present Cluster Contrast
which stores feature vectors and computes contrast loss at the cluster level.
Our approach employs a unique cluster representation to describe each cluster,
resulting in a cluster-level memory dictionary. In this way, the consistency of
clustering can be effectively maintained throughout the pipline and the GPU
memory consumption can be significantly reduced. Thus, our method can solve the
problem of cluster inconsistency and be applicable to larger data sets. In
addition, we adopt different clustering algorithms to demonstrate the
robustness and generalization of our framework. The application of Cluster
Contrast to a standard unsupervised re-ID pipeline achieves considerable
improvements of 9.9%, 8.3%, 12.1% compared to state-of-the-art purely
unsupervised re-ID methods and 5.5%, 4.8%, 4.4% mAP compared to the
state-of-the-art unsupervised domain adaptation re-ID methods on the Market,
Duke, and MSMT17 datasets. Code is available at
https://github.com/alibaba/cluster-contrast
- …